perm filename CONTRO[S86,JMC] blob
sn#818785 filedate 1986-06-07 generic text, type C, neo UTF8
COMMENT ⊗ VALID 00015 PAGES
C REC PAGE DESCRIPTION
C00001 00001
C00003 00002 contro[s86,jmc] The great Spring 86 AI controversy
C00004 00003 The invitation to the dance.
C00008 00004 The cast of characters.
C00010 00005 The play begins.
C00022 00006 Dreyfus:
C00041 00007 Winograd:
C00052 00008 Searle:
C00075 00009 vijay.ernie@berkeley.edu
C00078 00010 Hofstadter
C00106 00011 Weizenbaum
C00118 00012 Rumelhart
C00131 00013 ∂25-May-86 1849 vijay@ernie.Berkeley.EDU Weizenbaum's stage 3 comments
C00152 00014 ∂25-May-86 1851 vijay@ernie.Berkeley.EDU Dreyfus's stage 3 comments
C00159 00015 ∂31-May-86 1234 vijay@ernie.Berkeley.EDU AI DISC: Daniel Bobrow
C00177 ENDMK
C⊗;
contro[s86,jmc] The great Spring 86 AI controversy
The invitation to the dance.
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 3 Apr 86 15:14:58 PST
∂07-Feb-86 1529 vijay@ernie.berkeley.edu
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 7 Feb 86 15:29:14 PST
Received: by ernie.berkeley.edu (5.44/1.8)
id AA11470; Fri, 7 Feb 86 15:27:39 PST
Date: Fri, 7 Feb 86 15:27:39 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8602072327.AA11470@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Hello Dr. McCarthy,
I, Rajeev Aggarwal, and John Searle are doing a study which
is quite similar to the survey published by Daniel Bobrow in the
AI Journal this year. He will be helping us to make a publishable
version of this study for the AI Journal.
Basically, the whole study can be described/outlined in three stages.
In the first, we have three participants: Hubert/Stuart Dreyfus,
John Searle, and David Rumelhart. They have agreed to provide
approximate 2 page specific criticisms of traditional AI.
(Terry Winograd may also be participating, but this is not certain yet).
In the second stage, four computer scientists actively doing
work in the field will be providing responses to any parts
of the criticisms that they feel need to be refuted, based
on their work, other AI work, or their own philosophies. We
would very much like you to be one of the four participants
in this stage.
All the participants sincerely believe that your presence and views
are very important to such a discussion - for their own benefit and
the various readerships (publications) that we hope will see various
versions of this discussion.
In the last, third stage, we intend to get one last brief
response/comments from the critical side and then a final
statement from the AI researchers.
The exchange of communications will be organized in a manner
so that each participant will have a reasonable amount of time
to respond to other participants, one at a time.
If it is okay with you, we would like to conduct all communication
over the network since this will make the entire study go more
rapidly. We hope you will be able to participate and let
us know soon of your decision. We believe this will be
quite an interesting discussion!
Sincerely,
Vijay Ramamoorthy
The cast of characters.
∂13-Feb-86 1501 vijay@ernie.berkeley.edu
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 13 Feb 86 15:00:49 PST
Received: by ernie.berkeley.edu (5.44/1.8)
id AA03983; Thu, 13 Feb 86 14:59:05 PST
Date: Thu, 13 Feb 86 14:59:05 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8602132259.AA03983@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Hello Dr. McCarthy,
Thank you for responding so promptly; The complete list of
participants are John Searle, Hubert and Stuart Dreyfus, David Rumelhart,
Seymour Pappert, Joseph Weizenbaum, Eugene Charniak,
Douglas Hofstadter (in a "middle" position), Terry Winograd,
and yourself.
Next week we will be sending out complete information on
the discussion.
Sincerely,
Vijay Ramamoorthy
The play begins.
∂13-Mar-86 1941 vijay@ernie.berkeley.edu
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 13 Mar 86 19:41:18 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA13465; Thu, 13 Mar 86 19:42:23 PST
Date: Thu, 13 Mar 86 19:42:23 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8603140342.AA13465@ernie.berkeley.edu>
To: jmc@su-ai.arpa
***************************************************************
FINALLY --- IT'S HERE!!!
OUR AI DISCUSSION WILL BEGIN NEXT WEEK!!
***************************************************************
Thank you again for your participation, we hope that everyone
will benefit from having the advantage of putting forth their
ideas and receiving responses from such a diverse and dis-
tinguished group of people.
Following is some general information about the written dis-
cussion this study entails:
PURPOSES: We would like this discussion to be a free expression
of ideas about artificial intelligence. It will start with a
series of `critiques' on the traditional approaches that many AI
researchers have, and currently are taking. This will probably be
enough to provoke many responses against the criticisms, and then
responses to those responses. But it needn't always; agreement
is perhaps one of the best things to come out of any discussion,
and we hope that it will emerge in some form from this one. Par-
ticipants will have the consequence of sharpening their positions
and ideologies, and since this is a written discussion, everyone
will have the chance to get at the heart of the beliefs of others
- both by allowing time to think about certain ideas, and by be-
ing able to formulate responses without having to publish them
each time.
We also hope that "this meeting of the minds" will be a testing
grounds for new ideas/hypothesis to gain feedback from others.
There really isn't one sharp line that divides everyone, for al-
most no one agrees completely with anybody else anyway.
FRAMEWORK: There are 3 general stages to this discussion. The
first two will be somewhat formal, with the third being a general
"anything goes" informal exchange. They are outlined as follows:
Stage 1: This stage will consist of some criticisms on
current/traditional AI research; this is basically
to start the discussion; it will be given from group
one of the participants (as we have divided them)
to the other; the each of the criticisms will be
approximately 2 pages.
Stage 2: This stage will be the first response to these criti-
cisms; Each participant from group 2 will have the
opportunity to respond (support/agree or criticize)
anything in each of the critical papers - based on
their research, philosophies, or beliefs. These
responses will then be passed on to the group 1 par-
ticipants.
Stage 3: This last stage will partly build on the first two,
and be supplemented by whatever else comes up. Here
there will be rapid exchanges amongst the various
participants. Everyone will be able to monitor the
the discussion as it progresses.
PARTICIPANTS: This grouping really only applies to the first
2 stages; in the last, it is not important.
Group 1 Group 2
John Searle John McCarthy
Stuart/Hubert Dreyfus Daniel Bobrow
Terry Winograd Seymour Papert
Joseph Weizenbaum Eugene Charniak
In The middle:
Douglas Hofstadter
David Rumelhart
The division was not meant to be a major classification of any
type. It was arrived at based on past stances to traditional
information-processing oriented research. It's only purpose is
to provide part of a knowledge base/foundation for Stage 3.
One note about "In the Middle": for purposes of the first and
second stages, we decided to have Douglas Hofstatder and David
Rumelhart in a position where they will converse with both sides.
TIMETABLE: At the outset, we told everyone that there would be
"a reasonable amount of time to respond." This really applies
to the first two stages, where we would like to keep it to 2
weeks for the production of the first stage, and 2 weeks later
for the responses in the second stage. The third stage will
probably last several weeks, but this is generally open.
The time we have in mind for obtaining the criticisms of stage 1
is... FRIDAY, MARCH 21. At that time, we will pass all of the
papers on to all the group 2 participants. Two weeks from then,
we request all the group 2 responses to be in by FRIDAY, APRIL 4.
These responses will be forwarded to the group 1 members, and the
informal (stage 3) discussion will then begin (probably the most
interesting part). At that point, responses to specific people
will be forwarded immediately to the individuals involved. At
the end of each week, a transcript of the entire week's discus-
sion will be distributed to everyone.
COMMUNICATIONS: The entire discussion, as we have mentioned, will
take place entirely by electronic mail -- the fastest form of
written communication of this sort available to everyone. The
account that will be dedicated to handling all the communications
will be the following:
vijay@ernie.berkeley.edu
Once we start, all information will be processed immediately
after it is received. All messages received will be ack-
nowledged immediately and we hope that everyone will do the same
also. E-mail is reliable, but not "that" reliable.
PUBLICATION: Daniel Bobrow has been kind enough to offer his
help for collating multitudes of responses for publication in the
AI Journal. Furthermore, there will be a neutral introduction
and analysis to the entire discussion.
However, we will also be offering various editions of this dis-
cussion to various prominent national science publications. Our
philosophy here is that noting the quality of articles on AI, it
is clearly better that the current ideas driving AI research be
discussed by those directly involved with it, not by journalists
left to interpret it.
Furthermore, it almost goes without saying that everyone partici-
pating will receive a final copy of the sum total of all com-
munications that go on between the various participants in this
discussion.
Any further questions/problems, please forward them to this
account: vijay@ernie.berkeley.edu
Sincerely,
Vijay Ramamoorthy, U.C. Berkeley (Computer Science)
Rajeev Aggarwal, Bell Laboratories
John Searle, Dept of Philosophy, U.C. Berkeley
Daniel Bobrow, Xerox
(Project Organizers)
P.S. Remember, please acknowledge receipt of this message
through the account you would like us to send all your
responses/coments/information to.
Dreyfus:
∂01-Apr-86 1320 vijay@ernie.berkeley.edu AI DISC: DREYFUS
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 1 Apr 86 13:20:32 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA27558; Tue, 1 Apr 86 13:20:57 PST
Date: Tue, 1 Apr 86 13:20:57 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604012120.AA27558@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: DREYFUS
Hello Dr. McCarthy,
You might have seen a version of the following given by Dreyfus,
however, both Stuart and Hubert Dreyfus say that as a starting
point - it articulates their ideas most clearly.
------------------------------------------------------------------
CONVENTIONAL AI: A DEGENERATING RESEARCH PROGRAM
Looking back over 30 years, the field of conventional
rule-based AI appears more and more to be a perfect example
of what Imre Lakatos has called a degenerating research pro-
gram.[1] AI began auspiciously with Newell and Simon's work
at RAND. In retrospect, we see we failed to appreciate the
importance of this early work. Newell and Simon proved that
computers could do more than calculations. They demonstrated
that the symbols computers manipulate could stand for any-
thing, including features of the real world, and programs
could be used as rules for relating these features, so that
computers acting as logic machines could be used to simulate
certain important aspects of intelligence. Thus the
information-processing model of the mind was born. By 1970
AI, using symbolic representations, had turned into a flour-
ishing research program. Marvin Minsky, head of the M.I.T.
program, predicted that "within a generation the problem of
creating `artificial intelligence' will be substantially
solved."[2]
Then, rather suddenly, the field ran into unexpected
difficulties. The trouble started, as far as we can tell,
with the failure of attempts to program children's story
understanding. It turned out to be much harder than one
←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [1] Imre Lakatos, Philosophical Papers, ed. John Wor-
rall, Cambridge University Press, 1978.
$9 [2] Marvin Minsky, Computation: Finite and Infinite
Machines, Prentice Hall, 1967, p. 2.
- 2 -
expected to formulate the required theory of common sense.
It was not, as Minsky had hoped, just a question of catalo-
guing a few hundred thousand facts. The common sense
knowledge problem became the center of concern. Minsky's
mood changed completely in the course of fifteen years. He
told a reporter: "the AI problem is one of the hardest sci-
ence has ever undertaken."[3]
Related problems were also noted although not often
seen as related. Cognitivists discovered the importance of
images and prototypes in human understanding and logic
machines turned out to be very poor at dealing with either
of them. Gradually most researchers have become convinced
that human beings form images and compare them by means of
holistic processes quite different from the logical opera-
tions computers perform on descriptions.[4] Some AI workers
hope for help from parallel processors, machines that can do
many things at once and hence can make millions of infer-
ences per second, but if human image processing operates on
holistic representations that are not descriptions and
relates these representations in other than rule-like ways,
←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [3] Gina Kolata, "How Can Computers Get Common
Sense?", Science, Vol. 217, 24 September 1982, p. 1237.
$9 [4] For an account of the experiments which show that
human beings can actually rotate, scan, and otherwise
use images, and the unsuccessful attempts to understand
these capacities in terms of programs which use
features and rules, see Imagery, Ned Block, ed., M.I.T.
Press/Bradford Books, 1981. Also Ned Block, "Mental
Pictures and Cognitive Science," The Philosophical Re-
view, Oct. 1983, pp. 499-541.
- 3 -
this appeal to parallel processing misses the point. The
point is that human beings are able to form and compare
their images in a way that cannot be captured by any number
of procedures that operate on symbolic descriptions.
Another human capacity which computers functioning as
analytic engines cannot copy is the ability to recognize the
similarity between whole images. Recognizing two patterns as
similar, which seems to be a direct process for human
beings, is for a logic machine a complicated process of
first defining each pattern in terms of objective features
and then determining whether, by some objective criterion,
the set of features defining one pattern match the features
defining the other pattern.
As we see it, all AI's problems are versions of one
basic problem. Current AI is based on the idea which has
been around in philosophy since Descartes, that all under-
standing consists in forming and using appropriate represen-
tations. In conventional AI these have been assumed to be
symbolic descriptions. So common sense understanding has to
be understood as some vast body of propositions, beliefs,
rules, facts and procedures. AI's failure to come up with
the appropriate symbolic descriptions is called the common
sense knowledge problem. As thus formulated this problem
has so far turned out to be insoluble, and we predict it
will never be solved.
What hides this impasse is the conviction that the
- 4 -
common sense knowledge problem must be solvable since human
beings have obviously solved it. But human beings may not
normally use common sense knowledge at all. What common
sense understanding amounts to might well be everyday know-
how. By know-how we do not mean procedural rules, but know-
ing what to do in a vast number of special cases. For exam-
ple, common sense physics has turned out to be extremely
hard to spell out in a set of facts and rules. When one
tries, one either requires more common sense to understand
the facts and rules one finds or else one produces formulas
of such complexity that it seems highly unlikely they are in
a child's mind.
Theoretical physics also requires background skills
which may not be formalizable, but the domain itself can be
described by abstract laws that make no reference to
specific cases. AI researchers conclude that common sense
physics too must be expressible as a set of abstract princi-
ples. But it just may be that the problem of finding a
theory of common sense physics is insoluble. By playing
almost endlessly with all sorts of liquids and solids for
several years the child may simply have built up a repertory
of prototypical cases of solids, liquids, etc. and typical
skilled response to their typical behavior in typical cir-
cumstances. There may be no theory of common sense physics
more simple than a list of all such typical cases and even
such a list is useless without a similarity-recognition
ability. If this is indeed the case, and only further
- 5 -
research will give us an answer, we could understand the
initial success and eventual failure of AI. It would seem
that AI techniques should work in isolated domains but fail
in areas such as natural language understanding, speech
recognition, story understanding, and learning where the
structure of the problem mirrors the structure of our every-
day physical and social world.
In 1979 we predicted stagnation for AI, but also
predicted the success of programs called expert systems
which attempted to produce intelligent behavior in domains
such as medical diagnosis and spectrograph analysis which
are completely cut off from everyday common sense. Now we
think we were uncharacteristically over-optimistic concern-
ing the future of intelligent logic machines. It has turned
out that, except in certain structured domains where what
constitutes the relevant facts and how these facts are
changed by decisions is known objectively, no expert system
based on rules extracted by questioning experts does as well
as the experts themselves, even though the computer is pro-
cessing with incredible speed and unerring accuracy what are
supposed to be the experts' rules.
In our just published book Mind Over Machine we attempt
to explain this surprising development. We argue that
beginners in a domain are given principles to follow, but
most domains in which human beings acquire skills and
achieve expertise are, like everyday physics, domains which
- 6 -
do not lend themselves to being understood at an expert
level in terms of principles.[5] Therefore experts, as even
Edward Feigenbaum has noted, are never satisfied with gen-
eral principles but prefer to think of their field of exper-
tise as a huge set of special cases.[6] No wonder expert
systems based on principles abstracted from experts do not,
in unstructured domains, capture those experts' expertise
and so never do as well as the experts themselves.
We still think, as we did in 1965, that someday comput-
ers may be intelligent just as one day the alchemists' dream
of transmuting lead into gold came true. AI may be
achieved, however, only after researchers give up the idea
of finding a local symbolic representation of high-order
macrostructural features describing the world and turn
instead to some sort of microstructural distributed, holis-
tic representation that is directly amenable to association,
generalization and completion. If this is, indeed, the
direction AI should go, it will be aided by the massively
parallel machines on the horizon, but not because parallel
machines can make millions of inferences per second, but
because faster, more parallel architecture can better imple-
ment the kind of neurally inspired processing that does not
←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [5] Hubert L. Dreyfus and Stuart E. Dreyfus, Mind
over Machine, Free Press/Macmillan (1986).
$9 [6] Edward A. Feigenbaum and Pamela McCorduck, The
Fifth Generation, Artificial Intelligence and Japan's
Computer Challenge to the World, Addison-Wesley Pub-
lishing Company, 1983, p. 82.
- 7 -
use macrostructural representations of rules and features at
all.[7]
Hubert L. Dreyfus and Stuart E. Dreyfus
University of California, Berkeley
$9←←←←←←←←←←←←←←←←←←←←←←←←←
$9 [7] See for example D. Rumelhart and J. McClelland,
Parallel Distributed Processing: Explorations in the
Microstructure of Cognition, MIT Press/ Bradford Books,
1986.
Winograd:
∂01-Apr-86 1325 vijay@ernie.berkeley.edu AI DISC: Winograd Position
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 1 Apr 86 13:25:24 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA27764; Tue, 1 Apr 86 13:25:53 PST
Date: Tue, 1 Apr 86 13:25:53 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604012125.AA27764@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: Winograd Position
The best thing to do in a short position paper is to put forth some
clear and probably controversial assertions, without giving elaborate
motivations and justifications contrasting them with other ways of
understanding. These fuller discussions appear at length in my recent
book with Fernando Flores, Understanding Computers and Cognition.
1. In characterizing AI, there are two very different starting points.
We can take it as the general enterprise of developing intelligent
artifacts (by any physical means whatsoever), or as the expression of a
coherent methodology and theory.
2. To the extent that AI means "anything anyone might invent that shows
intelligence," discussion belongs in the realm of science fiction, since
there is little concrete to be said. To the extent we are talking about
what people have really done in AI, there is a strong coherent ideology,
variously labelled the "computational paradigm," "cognitive paradigm,"
"physical symbol system hypothesis," etc. Most of the existing AI
enterprise operates within it (including, though to a somewhat lesser
extent, the current work on connectionism).
3. The cognitive symbol-processing approach will have useful
applications, but these will not be as widespread or significant as
proponents claim. In general, those tasks that are closer to
"puzzle-solving" will be best covered, and those closer to "common
sense" and "ordinary understanding" will remain unmechanized. This
applies not only to existing technology, but to any of the foreseeable
improvements following in the general scientific direction that is being
pursued (including "massively" parallel machines, nonmonotonic
reasoning, etc., etc.).
4. I am not so concerned with the danger that attempts to fully
duplicate human intelligence will fail (as long as people don't bank to
heavily on optimistic predictions), but rather that the enterprise has
an effect of redefining intelligence---of shaping human understanding of
what is to count as "intelligent." In particular, AI is based on a
"rationalistic" account of human thought and language, which focusses on
systematic reasoning based on symbolic representations within an
explicitly articulated domain of features. This approach has important
uses, but systematically undervalues other aspects of intelligent human
action, both in the individual and within a tradition. Emphasis on
rationalism is not new to AI, having a long history in Western thought
(beginning with Plato, expressed more thoroughly by Descartes).
Computers (and AI in particular) give it a powerful operational form.
5. A healthy skepticism about AI (and the rationalistic orientation in
general) is needed as a guide for design of computer systems that make
sense. We are easily seduced by the image of the "thinking machine"
into claiming that the problems of designing and working with computer
technology will be solved when the machines get smart enough. The Fifth
Generation hoopla (both the Japanese original report and later books and
responses) is an egregious example of this fallacy. The phenomena of
"computerization" (in its pejorative sense) derive from the
reorganization of social systems to fit the properties of particular
computer implementations. It will not be prevented by having "smart"
machines, and in fact is accelerated by advocating the use of computers
in less structured areas of human life and society.
6. My major interest lies in research (both theoretical and applied)
that will support the development of technology to provide the
advantages of using computers while anticipating and avoiding negative
effects on people's work and lives. The rationalistic tradition does
not provide a sufficient basis for this design, since it takes as its
starting point an impoverished account of what people do. A new
starting point will come from an understanding of the phenomenology of
human communication and use of technology. We can draw on the
philosophical tradition of phenomenology, and its insights can be given
concrete operational meaning in the context of design.
7. It is often claimed that concerns of "social impact" should be left
to the political process, or perhaps to engineers who are directly
developing products, but should be ignored in pursuing "pure science."
These (often self-serving) claims are based on a rationalistic (and
narrow) understanding of science as a human enterprise. They might be
true for some idealized scientist living self-sufficiently and
incommunicado on an isolated island, but are irrelevant to the real
world. The continuing enterprise of any science depends on a public
consensus that supports the allocation of resources to it. This
consensus is maintained by a process of publication and "education" in
which the ideology of the science is promulgated and justified. As
members of the "AI community" we all participate in this, through
writing, talking, and teaching.
8. AI scientists and engineers have a responsibility to take their work
seriously---to recognize that both their inventions and their words have
a serious effect and to consider the effects consciously. The issue
isn't censorship, but positive action. It is useless to try to label
work that "shouldn't be done," but instead we can use our knowledge and
status to advance the things that "should be done," rather than just
those that "can be done." I anticipate a gradual shift of effort and
emphasis within the field as we go beyond the the early science-fiction
dreams that motivated the field, and look at directions for new research
(including theoretical research) that better deals with the realities of
human society. In particular, computers (using AI techniques) will be
understood in terms of the complex and productive ways in which they can
serve as a medium for human-to-human communication, rather than being
personified as surrogate people.
-TERRY WINOGRAD
Searle:
Received: by ernie.berkeley.edu (5.45/1.9)
id AA28407; Thu, 3 Apr 86 15:15:33 PST
Date: Thu, 3 Apr 86 15:15:33 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604032315.AA28407@ernie.berkeley.edu>
To: jmc@su-ai.arpa
TURING THE CHINESE ROOM
John R. Searle
Since various garbled versions of my Chinese room argu-
ment continue to be current in the CS-AI community, I intend
first to set the record straight. Then I intend to review
the current state of the argument concerning strong AI.
Among other things, I am accused of holding the prepos-
terous view that somehow in principle, as a matter of logic,
only carbon-based or perhaps only neuronal-based substances
could have the sorts of thoughts and feelings that humans
and other animals have. I have repeatedly and explicitly
denounced this view. Indeed, I use a variation of the
Chinese room argument against it: simply imagine right now
that your head is opened up and inside is found not neurons
but something else, say, silicon chips. There are no purely
logical constraints that exclude any particular type of sub-
stance in advance.
My actual argument is very simple and can be set out in
a very few steps:
Definition 1. Strong AI is defined as the view that
the appropriately programmed digital computer with the right
inputs and outputs would thereby have a mind in exactly the
same sense that human beings have minds.
It is this view which I set out to refute.
Proposition 1. Programs are purely formal (i.e. syn-
tactical).
I take it this proposition needs no explanation for the
readers of this journal.
Proposition 2. Syntax is neither equivalent to nor
sufficient by itself for semantics.
I take it Proposition 2 is a conceptual or logical truth.
The point of the parable of the Chinese room is simply to
remind us of the truth of this rather obvious point: the man
in the room has all the syntax we can give him, but he does
not thereby acquire the relevant semantics. He still does
not understand Chinese.
It is worth pointing out that the distinction between
syntax and semantics is an absolutely foundational principle
behind modern logic, linguistics, and mathematics.
Proposition 3. Minds have mental contents (i.e. seman-
tic contents).
Now from these three propositions, it follows simply
that strong AI as defined is false. Specifically:
- 2 -
Conclusion 1 : Having a program -- any program by
itself -- is neither sufficient for nor equivalent to having
a mind.
Anyone who wishes to challenge my argument is going to
have to show at least that one of the three "axioms" is
false. It is very hard to see how anybody in the AI commun-
ity would want to challenge any of them. In particular, the
idea that the program is purely formal and the computer is a
formal symbol manipulating device is hardly something that I
need to teach workers in AI .
Once you appreciate the structure of the argument it is
easy to see that the standard replies to it in the strong AI
camp are simply irrelevant because they do not address them-
selves to the actual argument. Thus, for example, the "sys-
tems reply" (according to which `the room,' i.e. the whole
system, understands Chinese even though the man in the room,
i.e. the CPU, does not understand) simply misses the point.
The system has no way of getting from the syntax to the
semantics any more than the man does. The systems reply
cannot evade the sheer inexorability of the syntax/semantics
distinction. Which axioms does it wish to challenge? And
what grounds are being given for the challenge? The "robot
reply" (according to which if we put the system inside a
robot capable of causal interactions with the rest of the
world it would thereby acquire a semantics) simply concedes
that strong AI is false. It admits that syntax would not
be sufficient for semantics but insists that syntax plus
causation would produce a semantics. This involves a
separate mistake that I will come back to, but right now I
want to emphasize that none of the defenders of strong A I
-- a rather large group by the way -- has even begun to make
an effective challenge to any of the three principles I have
enunciated.
That is the argument against strong AI. It is that
simple. Anyone interested only in knowing if strong AI is
false can stop reading right here. But now out of this sim-
ple argument gives rise to a whole lot of other issues.
Some of them are a bit trickier, but I will keep the argu-
ment as simple as possible. As before, the "axioms" must be
obviously true and the derivations must be transparently
valid.
If creating a program is not sufficient for creating a
mind, what would be sufficient? What is the difference
between the relation that mental states have to brains and
the relation that programs have to their hardware implemen-
tations? What are the relations between mental processes
and brain processes anyhow? Well, obviously I am not going
to answer all of these questions in this short paper, but we
can learn a surprising amount by just reminding ourselves of
the logical consequences of what we know already.
- 3 -
One thing we know is this: quite specific neurophysio-
logical and neurobiological processes in the brain ←λc←λa←λu←λs←λe
those states, events, and processes that we think of as
specifically mental, both in humans and in the higher
animals. Of course the brain, like a computer, or for that
matter, like anything else, has a formal level (indeed many
formal levels) of description. But the ←λc←λa←λu←λs←λa←λl powers of the
brain by which it causes mental states have to do with
specific neurobiological features, specific electrochemical
properties of neurons, synapses, synaptic clefts, neuro-
transmitters, boutons, modules, and all the rest of it. We
can summarize this brute empirical fact about how nature
works as:
Proposition 4. Brains cause minds.
Let us think about this fact for a moment. The fact that a
system has mental states and that they are caused by neuro-
physiological processes has to be clearly distinguished from
the fact that a system that has mental states will charac-
teristically behave in certain ways. For a system might
have the mental states and still not behave appropriately
(if, say, the system is human and the motor nervous system
is interfered with in some way) and it might behave in a way
appropriate to having mental states without having any men-
tal states (if, say, a machine is set up to simulate the
input-output functions of the human system without having
the appropriate mental states -- in a familiar example, the
system might emit the right answers to the right questions
in Chinese and still not understand a word of Chinese.) So
the claim that Brains Cause Minds is not to be confused with
the claim that Minds Cause Behavior. Both are true. But
the claim that brains cause minds is a claim about the "bot-
tom up" powers of the brain. It is a summary of the claim
that lower level neurophysiological processes cause, e.g.,
thoughts and feelings. So far it says nothing at all about
external behavior. Just to keep the distinction straight,
let us write this separate proposition as:
Proposition 5. Minds cause behavior.
Now with P. 5, unlike P. 4, we are not talking about bottom
up forms of causation. We are simply summarizing such facts
as that my pain causes me to say "Ouch," my thirst causes me
to drink beer, etc.
From P. 4 and P. 5 by transitivity of causation, we
can infer
Conclusion 2. Brains cause behavior.
But now with the clear distinction between P. 4 & P. 5
and the observation that the input-output relations of human
beings are mediated by mental states, we can see the real
- 4 -
power and implications of P. 4. The causal powers of the
brain consist not merely in the fact stated by C. 2, that
brains causes it to be the case that in response to certain
stimuli a person will emit certain outputs (e.g. someone
pinches me and I say "Ouch"). The claim is rather that
specific biochemical features of the brain by bottom-up
forms of causation cause all of our mental phenomena includ-
ing those mental phenomena that mediate input-output rela-
tions, i.e. those mental phenomena that cause behavior.
(E.g., when someone pinches me and I say "Ouch" it is
because I feel a pain, and the sensation of pain is caused
by neuron firings in the thalamus and the somato sensory
cortex.)
We have then a clear distinction between the causal
powers of the brain to produce mental states and the causal
powers of the brain (together with the rest of the nervous
system) to produce input-output relations. I certainly have
not demonstrated that P. 4 is true, but I take it that its
truth is demonstrated by the past century of neurobiology.
And in any case, does anyone really doubt it? Does anyone
really doubt that all of our mental states are caused by
low level (e.g.neuronal) processes in the brain? Now from
P. 4, it follows trivially that
Conclusion 3. Any system capable of causing minds
would have to have causal powers equivalent to the bottom-up
causal powers of brains.
This is a trivial consequence of P. 4. Conclusion 3 does
not tell us anything about how those causal powers have to
be realized. As far as logical possibility is concerned
they could be realized, as I have pointed out on numerous
occasions, in green slime, silicon chips, vacuum tubes, or
for that matter, old beers cans. I have also claimed that,
as a matter of empirical fact, the probabilities that beer
cans, silicon chips, etc. have the same causal powers as
neurons is, roughly speaking, zero. The chances that chemi-
cal properties of silicon chips will be equal in their
bottom-up causal powers to the properties of neurons is
about as great as the chances that silicon chips will be
able to perform photosynthesis, lactation, digestion, or any
other specifically biological process. However, as I have
said repeatedly, that is an empirical claim on my part, not
something to be established by philosophical argument alone.
But, once again, does anyone in AI really question it? Is
there someone in AI so totally innocent of biological
knowledge that he thinks that the specfic biochemical powers
of human nervous systems can be duplicated in silicon chips
(transistors, vacuum tubes -- you name it)? Frankly, I
doubt it. I think the underlying mistake comes not from
ignorance but from confusion: the confusion is to suppose
that the same input-output function implies the presence of
the same bottom up causation. This view is enshrined in the
- 5 -
Turing test, but a moment's reflection is sufficient to show
that it is false. For example, at an appropriate level of
description an electrical engine can have the same input-
output function as a gasoline engine -- it can be designed
to respond in the same way to the same commands -- but it
works on completely different internal principles. Analo-
gously a system might pass the Turing test perfectly, it
might have the same information processing input-output
functions as those of a human being and still not have any
inner psychology whatever. It might be a total zombie.
We can now see what was wrong with the robot reply. It
had the wrong level of causation. The presence of input-
output causation that would enable a robot to function in
the world ←λi←λm←λp←λl←λi←λe←λs ←λn←λo←λt←λh←λi←λn←λg ←λw←λh←λa←λt←λe←λv←λe←λr about the presence of
bottom-up causation that would produce mental states.
Now from these elementary considerations, we can derive
two further conclusions.
Conclusion 4. The way that brains cause minds cannot
be solely in virtue of instantiating a computer program.
This conclusion follows from Proposition 4 and Conclusion 1,
that is, from the fact that brains do cause minds, and the
fact that programs are not enough, we can derive Conclusion
4.
Conclusion 5. Any artifact that we design, any system
that is created artifically for the purpose of creating
minds, could not do it solely in virtue of instantiating a
computer program , but would have to have causal powers
equivalent to the bottom-up causal powers of the brain.
This conclusion follows from Conclusions 1 and 3.
Now in all of the vast amount of literature that has
grown up around the Chinese room argument, I cannot see that
any of my critics have ever faced up to the sheer logical
structure of the argument. Which of its axioms do they wish
to deny? Which steps in the derivation do they wish to
challenge? What they have done rather, like Hofstatder and
Dennett, is persistently misquote me or attribute views to
me which are not only views I do not hold, but views which
I have explicitly denied. I am prepared to keep winning
this same argument over and over again, because its steps
are so simple and obvious, and its "assumptions" can hardly
be challenged by anybody who accepts the modern conception
of computation and indeed our modern scientific world view.
It can no longer be doubted that the classical concep-
tion of AI, the view that I have called strong AI, is pretty
much obviously false and rests on very simple mistakes. The
question then arises, if strong AI is false what ought AI to
be doing ? What is a reasonable research project for weak
AI? That is a topic for another paper.
-------
-John R. Searle
vijay.ernie@berkeley.edu
responses
I found the three papers disappointingly insubstantial.
I have written out responses to all of them, but I think I'll
hold on to the responses to Searle and the Dreyfus's until
I return from the two week trip to Europe I'm starting on
Sunday. Searle's was the most fun, because it offers the
opportunity to respond to him with the same vigor with
which he treats those with whose opinions he disagrees.
I'm sending you the response to Winograd in the
hopes that it will induce him to overcome his laziness
and subject more of the material from his book to criticism.
Here is the response to the little Winograd wrote.
I would defend the "rationalistic orientation" against the
attack given in Flores's and Winograd's book, which I have read,
had Winograd bothered to present some of the attack. This defense,
however, would have to admit that some of the examples
in the book present problems for previous formalizations used
in AI. Their proper treatment requires a considerable elaboration
of the existing, though new, methods of formalized non-monotonic
reasoning. They may also require something along the lines of
formalized contexts, a subject I have recently been studying.
I especially like the question about whether there is
water in the refrigerator, the issue of what knowledge of flies
may be ascribed to a frog's retina, and the Heidegger (or is
it Flores and Winograd) parable of hammering.
Oh well, too bad.
As for the stuff about considering the consequences of
one's work, one should indeed, but the one must remember that
the scientist isn't the boss of society and can neither force
society to use the results of science nor prevent it from doing
so.
Hofstadter
∂09-Apr-86 1033 vijay@ernie.berkeley.edu AI DISC: Douglas Hofstadter
Received: from ERNIE.BERKELEY.EDU by SU-AI.ARPA with TCP; 9 Apr 86 10:32:58 PST
Received: by ernie.berkeley.edu (5.45/1.9)
id AA11863; Wed, 9 Apr 86 10:33:32 PST
Date: Wed, 9 Apr 86 10:33:32 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604091833.AA11863@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: Douglas Hofstadter
**** Please acknowledge receipt of this mailing -Thanks, VR
Impressions of AI
=================
Douglas R. Hofstadter
University of Michigan
April, 1986
I recently went to see an exhibition of Winslow Homer paintings, one
of which was called "Dog on a Log". The title amused me, and I invented
variations on it, such as "Frog on a Dog on a Hog on a Log in a Bog". While
doing this, I was thinking about how people -- especially small children --
love these sorts of simple rhyme, and about how when you hear such things, the
rhyme just jumps out at you, louder than the image behind it. Then I thought
about how speech-understanding programs work, and it occurred to me that if I
pronounced such a phrase into a microphone, a top-notch speech-understanding
program could understand exactly what I had said, in terms of getting all the
words right and putting together the right meaning -- but it would not hear
the rhyme. Not at all. And this seemed to me downright eerie. What would
it be like to hear that phrase PERFECTLY, but not to hear the rhyme? It
would be unimaginably alien -- like being dead, was my feeling.
Many traditional AI people would empathize with my human reaction, and
would say that the program should be fixed up to be able to hear rhymes.
So they would add a "rhyme-detection module", in essence. This does not
seem to me to get you one iota closer to being "alive", however. What about,
say, alliteration? Of course you can add on an alliteration-detection module
as well, and so on ad infinitum. But how many people really believe that this
is how brains work?
When I entered AI some ten years ago, my definition of the field would
have run something like this: "AI is the attempt to understand what thinking
is by making theories, implementing them as computer programs, and learning
from their performance how to improve the theories." Although I wish this
were true, it is far from it. Nowadays, aside from this philosophy, there
are at least two other common views of what AI is: (1) "AI is `knowledge
engineering' -- the field in which one tries to implant expertise in
computers"; and (2) "AI is the branch of computer science concerned with making
computers do more and more things that, for humans, involve intelligence
-- but its practitioners don't care whether the computers accomplish their
results in the way that people do." Both views are highly pragmatic, having
more to do with engineering than with science. Admittedly, excellent minds
can become engrossed in either type of approach, but the concern with the
workings of the human mind has been almost lost, and I believe that the human
mind is much more complex and subtle than anything that people can think
up themselves. I am firmly convinced that AI must first and foremost be
a cognitive science -- a science of the human mind -- rather than a type
of engineering, if it is to make real progress. Artificial intelligence
should be artificial only in the sense that a model of the mind is not as
complex as a real mind.
The engineering approach to building intelligence is a totally open
competition with any idea welcome, whereas in a scientific quest for how minds
work, off-base theories would ideally be pruned by experiments revealing their
flaws. However, since the field is so new, people have not devoted enough
effort to figuring out what constitutes a good test of a theory or of a
program. For instance, I once watched demos of some programs at Yale having
a most baffling property: they used tremendously intricate machinery to
mimic wonderfully flexible acts of cognition -- but those programs worked
with ONLY ONE INPUT. I was reminded of my one visit as a small boy to
Disneyland, where I took thrilling rides down jungle rivers in boats --
but I was most disappointed to find out that these "boats" were actually
running on tracks, so that they had totally fixed routes. (I hope that Roger
Schank would appreciate this reminding-incident.) What should one make of
programs that are so complex, so fancy, so rich with insights -- yet so
brittle? I certainly don't know.
Everybody admits that AI programs don't have common sense yet. One of the
most frequently suggested remedies to this lacuna is that we just give programs
MORE KNOWLEDGE. So one ambitious on-going project is to translate all the
articles in a huge encyclopedia into a sophisticated knowledge-representation
language. I can see the appeal of such a project, but I think it has nothing
whatsoever to do with minds or commmon sense. Adding huge amounts of knowledge
to current "inference engines" is a brute-force approach to a very subtle
problem, and I believe it will fail.
By contrast, I believe that the essence of common sense can be brought out
most clearly in tiny domains, somewhat condescendingly called "toy domains" by
many AI researchers (I like the term). A common view is that AI has exhausted
toy domains and should leave them behind, just as children exhaust the interest
of building blocks and go on to bigger things. I think this is absurd. One
can find huge mysteries in the tiniest of domains, devoid of practically any
world knowledge at all (and for which having vast amounts or world knowledge
would not be of any help).
To my mind, the quality of a scientific project depends critically on
whether it has identified and focused on some central problem in a very clear
way. Toy domains once played (and still could play) the role, in AI, that
idealizations of all sorts have always played in physics. Physicists think
of gases in terms of colliding billiard balls, or solids as perfect lattices;
they think of point particles moving in perfectly straight lines in a perfect
vacuum; and so on. The exact analogue in AI of such idealizations is perhaps
not clear, but certainly it is not a program designed to be an expert in a
highly arcane discipline that practically no one in AI really understands.
Domains like that are so complex that even a weak program can do some things
that people have not thought of. And, although it sounds simple-minded,
I think people are impressed when computers sling around technical jargon in
smooth natural-language discourse, so that weak programs look more awesome
than they are. In a tiny domain, you can't get away with that kind of thing.
Well-chosen toy domains therefore provide much more stringent challenges for
AI than huge domains do.
The canonical toy domain is the MIT "blocks world", home of such classic
AI programs as Guzman's and Waltz's vision programs, Winston's arch-learning
program, Winograd's SHRDLU, and Sussman's HACKER, among others. For reasons
unclear to me, this domain has lost favor in the AI world. People seem to be
under the impression that no challenges of interest could any longer be framed
in such a "small" domain. And yet by no means could any of the above-mentioned
programs be said to have been completed. They all had major defects and acted
very differently from people. They weren't integrated with each other. Simply
producing a blocks-world program that smoothly integrated the skills of ALL
the above-mentioned programs (in improved form) would be a phenomenally hard
task, and, to my mind, a wonderful accomplishment for AI.
As another example of a rich toy domain, I suggest the world of
letterforms -- "a" through "z". Each letter can be written in uncountably
many different ways, and the alphabet as a whole can be designed in uncountably
many different styles. Each letter defines a category that overlaps and rivals
other categories in amazingly subtle ways. Current optical-character-reading
technology has produced useful devices that allow a computer to do pretty well
in reading letters in many styles, but it affords no insight whatsoever into
the fundamental questions of categories, category boundaries, and analogy
(the underpinnings of a uniform visual style). Such devices are as far from
human visual perception as the rhyme-deaf speech-understanding programs I
mentioned above are from human hearing.
The letterform world is an ideal universe in which to study high-level
vision, the interface of perception with categorization, the structure of
very fluid categories, complex associations, overlaps, and rivalries of
categories, and highly abstract analogies, all of which I believe are
at the core of thinking. These issues have only rarely been approached
in traditional AI work -- and yet they are, to me, the problem of mind
in a nutshell. Most of the current work on perception is focused on
low-level (modality-specific) aspects, but I think that the high-level
aspects -- where perception merges with cognition -- are where the greatest
challenge and interest for AI lie. Traditional AI, with its strong focus
on natural language and deduction, tends to presume that all items come
pre-categorized (labeled with words). In such situations, serial models
of thought can do impressive things. On the other hand, such situations
represent but a tiny fraction of what real organisms in the real world
confront in real time.
Recently, there has been a healthy swing away from the serial-cognition
thrust of traditional AI; this movement is often called "connectionism",
although I think "emergent mentality" might be better. Connectionism is
based on the idea that there are fundamental things that serial computers
have shown themselves to be terrible at, and that require a totally different
approach. Years ago, AI people were appalled if you suggested that perhaps
they should be paying attention to how the brain works. They felt that
whole premise of AI was that thought has nothing to do with hardware.
Connectionism is a kind of backlash to that philosophy, and has its roots
in a number of interesting places: neurology, statistical mechanics and
thermodynamics, perhaps automata theory, and so on.
Connectionism's thesis -- that cognition is a collective phenomenon, in
which symbols and representation are statistically emergent phenomena rather
than directly accessible structures and processes -- is a subtle one, which
many old-guard AI people find upsetting, perhaps even inconceivable. Their
resistance is very understandable to me, since I believed in the old view
of AI for a long time. The old view is pretty much a consequence of the
following tight chain of near-equalities:
computation = logic
logic = reasoning
reasoning = thinking
If you believe these premises (and certainly each contains a grain of truth),
then you will have a hard time rejecting the obvious logical conclusion, to
wit:
thinking = computation
This is the basis of standard, mainstream AI. More precisely, that thesis
might be spelled out this way: "Thinking can be implemented as software
that runs on a von-Neumann-type serial machine".
I disagree, in varying amounts, with all three of the "equations" that
underlie this thesis. Actually, the first one is all right, provided you
think of "logic" as meaning merely "digital logic" ("TTL logic"). The third
one is ridiculous, in my opinion, but at least I find it easy to say why: I
have come to believe that most of thinking has nothing to do with reasoning
at all, but with a kind of associationism in which each concept has a hard core
surrounded by a soft blur that cannot possibly be considered part of reasoning.
Typos, speech errors, and many other sorts of error are among the consequences
of that blur; so are clever jokes, beautiful analogies, and intuitive leaps
that result in scientific discovery. The trickiest equation of the three is
the middle one, where the left-hand side ("logic") means two very different
things at once: both "digital (TTL) logic" and "mathematical logic", while
the right-hand side ("reasoning") means "sensible thought pattterns". (This
confounding of meanings of one word is typical of the blurry quality of human
thought -- and in this case it leads to error.) This equation hides perhaps
the subtlest error; because AI people so revere reasoning and because computers
have such a beautifully "logical" internal architecture, one simply WANTS to
equate the two.
All connectionists find fault, for one reason or other, with this
chain of "equations", and believe that the individual micro-elements of
the substrate from which genuine thought (human or artificial) emerges need
not have any representational quality; that that quality attaches only to
large-scale emergent aspects of such a substrate, much as life is a property
of an organism but not of its individual molecules. I am very glad to see
such a school begin to flourish, and I expect many fundamental ideas to
emerge from connectionism over the next decade or two.
My only qualm about this new approach to modeling the mind is that
it so totally eschews AI's original vision of cognition as explicit serial
manipulation of meaning-carrying symbols that it will not provide a framework
in which to naturally address the epistemological questions that AI has been
trying to answer. The point is that AI has always attracted minds that are
interested in how MINDS work, not necessarily how BRAINS work. While it is
probable that mentality is a kind of emergent phenomenon, so that you have
to devote some time to studying its non-thinking micro-components, you have
to beware that you don't get so absorbed in the study of the substrate that
you totally forget about thinking itself.
Connectionist models run the risk of becoming huge simulations that
perform impressively but have little explanatory power. An analogy may
help. Astrophysicists have constructed impressive computer models of the
evolution of galaxies, with thousands of simulated stars interacting via
simulated gravitation. Some of the unexplained properties of galaxies have
been reproduced, such as the formation of spiral arms -- but no one really
understands why. This is a wonderful result, but an explanatory bridge
between the micro- and macro-levels has not been built.
Cognitive psychologists have in essence asked themselves "What is a
concept?" That is, to my mind, the single most important question that AI
ought to be trying to answer. Connectionism may provide an important part
of the answer to this question, but I do not think it will do so alone. Some
insights at the level of thoughts themselves, not just their substrate, will
be essential. Although in some ways, the following will be a shaky analogy,
I would like to suggest it, because it contains a grain of truth. The best
traditional AI (and cognitive psychology) is something like classical physics:
true on a large scale, false on a small scale. Connectionism is something
like quantum mechanics, the ultimate basis of all physics: true on all
scales, but in some sense "irrelevant" on a large scale. The problem
is therefore to link these two vastly different levels. In physics, the
"correspondence principle" says that the equations of quantum mechanics
must turn into their classical counterparts in the limit of large quantum
numbers. In that sense, a beautiful bridge is rigorously established between
the unfamiliar micro-world and the familiar macro-world. I would like to see
such a bridge established between connectionism and the study of cognition
itself, which includes traditional AI, cognitive psychology, linguistics,
and the philosophy of mind. The ultimate goal, to my mind, would be to
provide in this way a hierarchical explanation for such high-level theoretical
constructs as the id, the ego, and the superego (or something along those
lines), allowing us finally to locate in brains or machines the "soft
architecture of the soul".
AI is much more of a hodge-podge than I had expected it to be when I
entered the field. Because it is a fledgling science, and because so much
is unknown, a hundred flowers are blossoming -- but many of them will wilt
rapidly. No one can yet be sure that a particular program or a particular
school or a particular approach is really on the right track to explaining
minds, because most achievements, however grand they may appear on the
surface, are still only tiny fragments of a huge puzzle. It is vitally
important for AI people to resist grandiose claims, and to stress the
limitations of their models. When AI people learn to resist the allure
of complexity, I think they will have taken a major step forward.
Weizenbaum
∂15-Apr-86 2331 vijay@ernie.berkeley.edu AI Disc: Joseph Weizenbaum
Received: from [128.32.0.6] by SU-AI.ARPA with TCP; 15 Apr 86 23:31:29 PST
Received: by ernie.berkeley.edu (5.45/1.12)
id AA14126; Tue, 15 Apr 86 23:32:47 PST
Date: Tue, 15 Apr 86 23:32:47 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604160732.AA14126@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI Disc: Joseph Weizenbaum
About AI
One of the definitions of AI that floats around the computer environment is
that it is the intelligence exhibited by computers when they do things
which, were they done by people, would be considered evidence of the
performer's intelligence. One trouble with this definition is that it is
too broad. It would cover, for example, virtually the entire set of
applications and systems programs ever written. It would, after all,
require considerable intelligence on the part of human beings to solve
systems of differential equations, to invert matrixes, or to manage the
execution of jobs under severe constraints of time, priorities, etc.
Computers do all these things, yet, systems and applications programmers
don't think of themselves as working in AI. Moreover, the "AI community",
a social category on whose definition there is little, if any, consensus at
all, specifically excludes such computer professionals. That this
exclusion is arbitrary can be seen from the fact that literally no one
suggests that the computer systems that routinely land wide body aircraft,
such as Boeing 747's and DC-10's, a task that surely requires great
intelligence on the part of human pilots when they take the controls, no
one suggests that these systems are in the category AI. Ed Feigenbaum,
who, by the way, is emphatically thought not an AI worker by, for example,
the AI insiders at the MIT and Yale Universities' AI laboratories, excluded
MACSYMA, the spectacularly successful applied mathematics system developed
by Joel Moses at MIT concurrent with the developments of Stanford's
DENDRAL, from his list of existing "expert systems" published in his Fifth
Generation, on the ground (so he told me personally) that MACSYMA is not
AI!
Even though the above quasi definition of AI is unsatisfactory in many
repects, it is useful. I prefer it, at least in the present context, to
definitions that appeal to "the way humans do things" or to analogies to
the human brain and such. Under this definition, exercised with a little
bit of common sense and judgment, today's chess playing machines are in,
even though they don't choose (compute) their moves in the same way, or
even in a way resembling the way, chess masters choose their moves. Also
in are robots whose behavior is in part a consequence of their perception
of their environments by means of sensors such as video cameras, touch
sensitive fingers, and so on. Cruise missiles are in. So, of course, is
the set of computer systems developed under the heading "Cognitive
Science".
The reason I prefer to be that all inclusive is that, while I have no
quarrel at all with the idea that there is such a thing as artificial
intelligence, that there exist some, as seen from a performance point of
view, very impressive examples of it, I do not agree with the claim that
"the [computer's] ability to [think and learn and create] is going to
increase rapidly until - in the visible future - the range of problems they
can handle will be coextensive with THE RANGE TO WHICH THE HUMAN MIND HAS
BEEN APPLIED", as Herb Simon wrote almost 30 years ago - in 1958, to be
exact, that is, in the almost no longer visible past.
The artificial intelligencia and I have never disagreed about 1) that
artificial intelligence is possible, 2) that there is no way to put a limit
on the degree - that is, the amount or magnitude - of AI machines may in
future achieve. For all I know, machines may someday be "more intelligent"
than their human models or creators. In any case, I don't know how such an
assertion can be disproved - even apart from the necessary vagueness of its
language. This, by the way, appears to be a point on which Dreyfus and I
disagree. My point of departure from the artificial intelligencia is that I
insist a machine's intelligence, no matter how "great" - must always be
different from and, in certain very important respects, alien to human
intelligence, whereas they, the people claiming to represent AI, believe
that machines can be made to think as humans think, that they can be made
to understand human language as humans do, that, in other words they can
achieve, if that's the right word, an inner life no different in all but
trivial ways from the inner life of human beings.
My belief that machines cannot be made to understand human situations
essentially involved with such matters as respect, love, the psychological
identity (in Eric Ericson's sense, for example) of individuals, and so on,
leads me to the derivative belief that there are areas of human concern to
which AI machines OUGHT not to be applied even if, perhaps especially if,
such machines can be made to ask and answer questions, perhaps in natural
language, in those areas. The machine responses may well create a very
powerful impression, i.e., the illusion, that the machine "understands", it
must, however, reach its conclusions on bases which human beings ought not
to accept in such human problem situations. Machines ought not, for
example, practice clinical psychiatry - so-called talk therapies - nor act
as judges or jurors in human conflict resolution efforts. This conclusion
is not at all sensitive to the manner of achieving AI, that is, whether AI
is realized on Von Neuman machines, connection machines, machines designed
to duplicate the neural (or whatever) structure of the brain. Individual
human beings are the products of their biological constitution and of their
histories. No artifact can have had a history of experiences remotely
resembling that of a human being. Nor can the history of any individual
human being be expressed by a finite bit string (which could be given to a
computer in lieu of its having the human experience itself). I think these
points are obvious. Those who disagree with me on these fundamental points
are free to believe that here, if anywhere, is an outcropping of faith,
theology, belief or whatever, in other words of something that they believe
need not be taken seriously in a discussion of scientific matters.
That the computer is and can be a powerful metaphor, in terms of which much
of the functioning human can be discussed and, in a certain sense,
understood goes without saying in this ninth decade of the 20th century.
Consistent with the position I have here sketched, however, I think there
is more to the life of the mind (at least mine) than cognition. An
understanding of the human mind resting entirely on the computer metaphor
must, in my view, be necessarily grossly incomplete. That it may
nevertheless be useful, even very useful, I don't dispute.
Of those, however, who believe that the improvement in computer chess
playing over the last dozen years or so is a triumph of cognitive science
and not due mainly to the increasing raw power of the computers involved, I
would like to ask what new psychological principles have been discovered
and deployed to account for the strength of todays machine chess play.
-------
- Joseph Weizenbaum
Rumelhart
∂16-Apr-86 2229 vijay@ernie.berkeley.edu AI DISC: David Rumelhart
Received: from [128.32.0.6] by SU-AI.ARPA with TCP; 16 Apr 86 22:28:00 PST
Received: by ernie.berkeley.edu (5.45/1.12)
id AA08440; Wed, 16 Apr 86 22:29:34 PST
Date: Wed, 16 Apr 86 22:29:34 PST
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8604170629.AA08440@ernie.berkeley.edu>
To: jmc@su-ai.arpa
Subject: AI DISC: David Rumelhart
(Here is a brief statement.)
Since I suspect different discussants have different conceptions of what
Artificial Intelligence is, it will be worth while for me be as explicit as I
can about my view of the enterprise. AI, like many other fields, can be
separated into three distinct, but interrelated activities. I would
characterize these as (1) theoretical AI, (2) experimental AI and (3) applied
AI. I will sketch below my basic understanding of these three activities.
Theoretical AI
I take theoretical AI to be a branch of mathematics -- a branch devoted
to the development of mathematical systems inspired by considerations of human
cognitive behavior. The activities of the theorist generally involve the
development of formalisms which are thought to be appropriate for expressing
algorithms capable of human-like behavior. Of course, as in all mathematics,
such algorithms and formalisms are not true or false. They are merely useful
or unuseful for some particular task. In this sense AI is not a science any
more than mathematics is a science. There is no empirical domain against
which AI should be held accountable. Of course, psychologists (or AI
practitioners acting as psychologists) may well select some of the
representations and algorithms developed by theoretical AI and state
psychological theories in terms of those formalism. Such theories, like all
scientific theories are then subject to empirical verification. I will say more
about using AI formalisms for stating psychological theories below.
Experimental AI
AI differs from other branches of mathematics in that they do not
ordinarily justify their work on the basis of theorems they might
prove, rather the justification is normally done by the development of computer
programs of one kind or another which are designed to explore and evaluate their
ideas. In this sense, AI is empirical -- do the algorithms do what they were
designed to do? Are there better ones for the kinds of tasks practitioners
have in mind etc? For these reasons computer programs play a unique and
important role in informing and justifying work in AI. They, of course, also
play an important role in giving the AI practitioner a unique perspective on
cognition and therefore, indirectly, shape the kinds of work done by
theoretical AI.
Applied AI
Applied AI is the application of the techniques and formalisms
developed by theoreticians to real problems. There is a fine line here between
applied AI and software engineering in general. AI techniques may be employed
to do whatever tasks computers might be called upon to do. Expert systems are
the most common applications today, but robotics, image analysis, speech
recognition, computer aided instruction and other possibilities are areas where AI techniques can and have been developed.
Cognitive Science
If you are a Cognitive Scientist interested in stating theories
about how people function, it is natural to look to AI as a source for
hypotheses and formalisms within which to state our theories. This, I
think, is because the kinds of mathematics that were developed for the
purposes of expressing theories in the physical sciences have not seemed
to carry over very well to cognitive science. In any case, the
past fifteen years or so have seen an upswing in the use of AI concepts
in accounting for human cognitive processes. However, this does not
necessarily mean that the formalisms that AI has already developed are
the ones which, in the end, will be appropriate or useful. It does
seem clear enough that many of the ideas which dominate modern AI have
been useful in taking a first cut at describing human cognitive
processes. It has seemed to me, however, that many of the ideas from
AI have been a bit misleading and have led people to propose theories
which are probably dead ends. (Note, this does not reflect on
theoretical AI, only on its use for expressing psychological models.)
AI systems have proven to be notoriously brittle and rigid. It seems
to me that this is largely due to the fact that the dominant conception
of computation in AI is based on our understanding of how conventional
computers work. My colleagues and I have become increasingly convinced
that the symbol processing architecture that has served AI more or less
since its inception, probably will not be of as much use in the
description of human cognitive processes (especially those which take
place in fractions of a second) and that we should look elsewhere for
our inspiration for building accounts of cognition. Of course, AI's
tie to symbol processing as the basic paradigm is only historical.
Indeed AI practitioners over the years have considered alternative
computational systems and today a number of AI practitioners are actively
pursuing alternatives.
Parallel Distributed Processing
Parallel distributed processing or PDP for short is the name we give to
the particular brand of "connectionist" models that I see as providing a more
powerful and appropriate formalism within which to formulate models of
cognition. The basic idea is to attempt to give a very general
characterization of what we call "brain style" processing and then to develop
models within that framework. What we have found is that the algorithms most
appropriate for the brain-style architectures are very different than those
appropriate for symbol processing type systems. Moreover, we find that these
algorithms which are natural for PDP models seem better suited as models of
human information processing. They exhibit a number of useful properties
including: ability to rapidly solve a very general "best match" problem,
graceful degradation (rather than brittleness), ability to formulate simple,
powerful learning procedures which let them readily adapt to their environment,
ability to allow a very large number of mutual "soft constraints" to be quickly
and efficiently taken into account in finding the best match, the ability to
naturally represent similarity and automatically generalize to new situations,
naturally form prototypes and a number of other similar features which
seem important in accounting for human cognitive processing. Of
course, the development of these ideas is still in its infancy. It
will probably turn out that some of the features which we now think are
important will not, in the end, turn out to be and features of brains
which we are currently ignoring will turn out to be critical. Still,
we believe that the general strategy of looking directly at brains and
asking about the nature of brain-style processing will ultimately prove
to be a valuable strategy in our understanding of the nature of human
intelligence. Moreover, we suspect that developments in PDP style computation
will ultimately contribute to AI and, to a certain degree, liberate AI from the
historical grip of logic based and symbolic processing systems and encourage
the search for other more appropriate architectures.
D. E. Rumelhart
Institute for Cognitive Science
University of California, San Diego
vijay@ernie.berkeley.edu
Comments on Rumelhart
Copyright 1986, John McCarthy
In general I have only minor quibbles with Rumelhart's classification of
AI related activities. Whether PDP style architectures will win out over
logic based architectures or whether they will be combined is something
for the future to tell us.
vijay@ernie.berkeley.edu
request for acknowledgment
Please acknowledge receipt of the following messages:
general position statement and separate messages on the Dreyfus's, Searle,
Winograd, Hofstadter, Weizenbaum and Rumelhart.
∂25-May-86 1849 vijay@ernie.Berkeley.EDU Weizenbaum's stage 3 comments
Received: from ERNIE.Berkeley.EDU by SU-AI.ARPA with TCP; 25 May 86 18:49:28 PDT
Received: by ernie.Berkeley.EDU (5.51/1.12)
id AA02562; Sun, 25 May 86 18:49:20 PDT
Date: Sun, 25 May 86 18:49:20 PDT
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8605260149.AA02562@ernie.Berkeley.EDU>
To: jmc@SU-AI.ARPA
Subject: Weizenbaum's stage 3 comments
Preface:
I am well aware that readers of these remarks - other's, as well as mine -
expect a discussion of 'scientific' and 'technical' issues, such as
speculations about the role of mathematics in AI or whether connectionism
will lead to machines exhibiting self awareness, and so on. As I
understood the invitation to participate in this forum, each of us was to
discuss whatever he thought was important and interesting about AI. I
think there are two matters related to AI that overshadow all others in
importance.
One has to do with what awaits us if we succumb to the temptation so
seductively dangled before us by, I would say, irresponsible or vastly
ill-educated AI practitioners, the temptation to believe that artificial
intelligence can be, or indeed is, equivalent to human intelligence in the
sense that it can understand and produce wisdom with respect to
interpersonal, social and cultural human affairs, as occasionaly human
intelligence can.
The other matter of overriding importance is that the work which is or is
paraded as being part of AI, whether correctly or not, has become an
indispensable component, even one of the foundations of the military
armamentarium which threatens the very survival of the human race. The
crucial role the military has assigned to AI in such programs as the
Strategic Defense Initiative (SDI) and the Strategic Computing Initiative
(SCI) casts a mantle of responsibility over the entire AI community. It is
now inescapably clear that the insane pace of the technological arms race
cannot be maintained without the willing cooperation of virtually the
entire AI community, academic and industrial. (Most of the rest of the
computer community is not exempt from this judgment. We happen here to be
concerned with AI.) In other words, the AI community has the power to
brake the mad cycle. It can rightly say "Not without us !" but
does not say it. We must all live with our consciences.
You may say we can't just discuss these two points forever. Correct. But
we can't evade them forever either.
End of preface.
McCarthy's two pointers, one to his long ago review of my book CP&HR and
one to something Berliner wrote in who knows what journal, don't help me
very much. True, there are things about which I haven't changed my mind.
Well then, JMC still refers to papers he wrote in 1958 and book reviews
over a decade old. There is some merit in consistency.
I reiterate :
1) I see no way to put a limit on the degree of intelligence artifacts may
eventually gain.
2) I am convinced that machine intelligence must always be alien to human
intelligence and that
3) the necessarily alien nature of machine intelligence has some
implications with respect to what we ought and ought not to do with machine
intelligence.
My disagreement is with those who insist that machines can be made to
understand human events in human terms. I have never understood, and
perhaps one of you can help me here, what it is that the true believers in
AI cannot accept in the above propositions.
To Hofstadter I want to say that I don't know what the phrase "the human
soul is infinite" means, hence whether it corresponds to anything I
believe. I do think that there are human experiences, lots of them, that
cannot be fully expressed by bit strings of any length. There are, in
other words, things humans know that they cannot say. Is it really the
position of mature AI researchers that that isn't true ? That it isn't
true of them as individuals ? That they can say everything they know? Can
Hofstadter express the whole of his experience of hearing a rhyme in any
way at all, in writing, for example ? If one of you has a dream so
beautiful that you want to hold on to it, perhaps to sleep again and
continue it, can you write it down or tell it to someone and know that you
have captured all of it ? Doesn't the very attempt to hold on to a dream
destroy it ? (Am I unique in this respect !?) Can even the events of a
single day be recorded completely - let alone the whole of a person's
history. That is why we have art - art is the attempt to transcend such
boundaries and, in the absolute sense, it is doomed to failure in that
attempt. Not so ?
All of this says that there are limits to what a non-human organism (if you
will) can be made to understand, and it says something about the nature of
these limits. If I were to argue anlogously with respect to Dolphins,
would anyone disagree ?
Natural language understanding is the arena in which issues of this kind
arise in very sharp form. Perhaps if and when computer vision has
progressed very far, scene understanding will become another such arena.
(To what extent will machines ever understand the look on that Frenchman's
face as he watched German troops march into Paris, as it was recorded in
that famous photograph ? Or the scene of two naked children, their backs
covered with burning napalm, running toward the camera in another famous
photograph ?) But, back to language. There seems to be general agreement
that understanding of an utterance depends on the understander sharing the
speaker's (or writer's) context.
JMC and others often make a distinction between "full understanding" and
"useful lesser levels of understanding." I would make the same
distinction. The reason we can make ourselves understood to, say, a Tokyo
(or, for that matter, New York) cabbie is that our very situation, e.g.,
having just hailed the cab and climbed in, together with the purpose of our
attempt to communicate, so drastically restricts the contextual framework
that communication amounts to little more than engaging in a very short
multiple choice test. Yes, such lesser levels of understanding are useful
and, I have no doubt, we can get computers to understand the kind of things
one says to waiters, cabbies, and so on, in the ordianry run of doing
business with them. But it doesn't follow from that there exists a
'default context', i.e., a context in which the sentence to be understood
has exactly one correct interpretation, a "literal meaning". History
enters into context formation. There are no 'standard conditions' that
apply to human affairs. What is given up when one accepts that ?
Dan Dennet coined the very telling term 'Cognitive Wheel'. He points out
that wheels do not occur in nature, that human beings invented wheels and
that this 'unnatural' invention has permitted human beings to do better
than nature in several respects. He suggests that artificial intelligence
may well come up with ways to "think" (why not ?) very different from the
way brains think and, in many respects, in ways superior to human thought.
I can't prove otherwise. Whether or not the attempt should be made is a
question of research priorities. But, whatever the decisions, it seems to
me important that questions of what can and what cannot be done be
confronted in earnest and soberly. Perhaps that is in part what we're
doing here.
A few quotes:
DH: "Scientific discoveries have fantastic potential in either
direction, bad or good, and so I would not curtail science in any
way."
JMC: "The scientist isn't the boss of society and can neither
force society to use the results of science nor prevent it from
doing so. This divison of labor between science and politics is
socially apporpriate."
DH: "...AI as a scientific activity is quite benign, and
therefore [I] feel that this forum ought to concentrate on AI as
science rather than on AI as social activity."
TW: "It is often claimed that concerns of 'social impact' should
be left to the political process, or perhaps to engineers who are
direct;y developing products, but should be ignored in pursuing
'pure science.' These (often self-serving) claims are based on a
rationalistic (and narrow) understanding of science as a human
enterprise. They might be true for some idealized scientist
living selfsufficiently and incommunicado on an isolated island,
but are irrelevant to the real world. The contiuing enterprise
of any science depends on a public consensus that support the
allocation of resources to it. This concensus is maintained by a
process of publication and 'eductaion' in which the ideology of
the science is promulgated and justified."
I agree with the last quotation, of course. But I would add that science
is a social enterprise in another sense as well. What is and what is not
to be counted as scientific, as fact is decided by a consensus of members
of the relevant section of the scientific community. Who is and who is not
a member of a particular section is similarly a social decision. It makes
a difference when someone who was once out is, as the communists say,
rehabilitated. (And vice-versa. A third of the members of this panel have
reasons to know about that.) Whether or not it is a legitimate AI task to
attempt to create a computer system that will take over the office of the
President of the United States under certain conditions, or to work on an
AI based expert system to do, say, astrological fortune telling, these are
questions answered by socially arrived at consensus.
The first three quotations above present the doing of science and science
itself as value free. They assert that the products of science can after
all be used for good or evil. Which way they are used is somebody else's
department. That's the socially appropriate division of labor. Perhaps
so. Perhaps on that isolated island Winograd mentions, scientists just
couldn't predict to what end use, good or evil, their work would be
put.vil. But we live in a concrete society which has a long and vivid
record of co-opting virtually every scientific and technical achievement of
the human genius to the purpose of creating and deploying ever more
efficient killing machines. (Forgive me for not using the usual
euphemisms.)
Some of us even celebrate that fact as, for example, when they tell us
that, if society generously supports AI and an American 5th Generation
effort to match Japan's, the so called 'smart weapons' of today will seems
like simple 'wind-up toys' compared to what we will then be able to
produce.
We know very well to what end uses, say, advances in computer vision, or
speech understanding, or robotics, or ... and so on and on, will be put.
In the concrete social and political reality in which we live, scientists
and engineers who work to improve the computers' ability to see, for
example, tacitly give their assent to the use of their work to help guide
nuclear tipped cruise type missiles on their way to mass murder. Except
for acts of self deception of heroic porportions, we know. We cannot then
deny our responsibility by arguing that our work is neutral, our science
value free, that somebody else decided, and so on.
I disagree with those who believe it necessary to maintain present levels
of weaponry, support bases overseas, and so on. But I don't think holding
that belief necessarily involves an abdication of responsibility. I do
think, on the other hand, that scientists and technologists who take the
position with respect to their work that
1) they don't want to contribute to weapon technology, and
2) that, while they know that their work can be used in ways they don't
like, it could equally well be used benignly,
are gravely mistaken. Science and politics simply can't be disentangled.
Responsibilities can't be shed by simply closing one's eyes. It just isn't
possible to discuss AI and what's to become of it without looking at and
deciding about the context in which it is embedded - not, that is, without
a constant effort of self censorship.
Finally, I didn't suggest that current chess playing programs don't embody
psychological principles. What I asked was what new psychological
principles have been discovered and deployed THAT ACCOUNT FOR THE STRENGTH
of today's machine chess play. I mean, of course, that account for
improvement beyond that brought about by the increase in raw computing
power deployed these days. Perhaps Berliner's paper answers this question.
I'd like to be able to look.
Joseph Weizenbaum
-------
∂25-May-86 1851 vijay@ernie.Berkeley.EDU Dreyfus's stage 3 comments
Received: from ERNIE.Berkeley.EDU by SU-AI.ARPA with TCP; 25 May 86 18:50:56 PDT
Received: by ernie.Berkeley.EDU (5.51/1.12)
id AA02577; Sun, 25 May 86 18:50:56 PDT
Date: Sun, 25 May 86 18:50:56 PDT
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8605260150.AA02577@ernie.Berkeley.EDU>
To: jmc@SU-AI.ARPA
Subject: Dreyfus's stage 3 comments
Dreyfuses to John McCarthy
You raised interesting questions concerning our views
on how the brain detects similarity and how typical situa-
tions are constructed.
Until recently, we had no clear idea about how the
brain recognizes the similarity of a current situation to
some stored memory, but the phenomenology led us to believe
that it was somehow done in terms of stored records of brain
activity patterns rather than in terms of abstract symbolic
descriptions. Recent connectionist work has provided us
with a reasonable explanation at the brain level quite dif-
ferent from what we had in mind, although this explanation
too is no doubt wrong in its details. Rather than store
brain-records of memories (of which one would presumably
need a long list), each experience (sensory input and an
associated output that experience shows to be appropriate)
leaves behind a "trace" by affecting the strengths of con-
nections between neurons. These strength modifications occur
in such a way that the input will tend to produce the
appropriate output when the neuron net "does its thing."
Then, later, the sensory input to the brain in some then-
current situation produces an output based on these connec-
- 2 -
tions. When the input is similar to one or more previously
seen inputs, the output will be similar to the appropriate
output due to the memory trace (i.e., appropriate connection
strengths).
Hence, memories themselves are not stored, nor are typ-
ical ones created. Many similar experiences merely produce
certain strong connections that then reproduce the appropri-
ate learned output when presented with a new, but similar,
input.
This account was motivated by Chapter 17 of Rumelhart's
and McClelland's book Parallel Distributed Processes, MIT
Press, May 1986.
While the input and output may, in some sense, be
interpretable as meaningful facts and beliefs about the
problem domain, if the neural net has hidden nodes and feed-
back among them, the process of getting from input to output
by "settling" of a neuron net will result in changing pat-
terns of activation of hidden nodes that have no interpreta-
tion in terms of facts and hypotheses about the problem
domain, and the strengths of connections among hidden nodes
have no interpretation as correlations or inferences con-
cerning any domain-meaningful facts. This is the sense in
which we believe that a process at the brain level need have
no correlate at the level of symbol manipulation where the
symbols have meanings concerning the domain and these mean-
ings are taken account of in the manipulation process.
- 3 -
While there may be meaningful symbol-manipulation
interpretations of certain inferential activities, mainly
those of non-experts, we believe that most expertise is of
the associative, pattern completion, type that can be
explained only at the brain (neuron net) level.
Dreyfuses to Hofstadter
We do indeed believe that expertise is largely the pro-
duct of the people, not leaders, to use your governmental
metaphor. See our note to John McCarthy. The fact that
this belief is clearly untenable concerning governmental
structure implies nothing about its correctness for cogni-
tion.
∂31-May-86 1234 vijay@ernie.Berkeley.EDU AI DISC: Daniel Bobrow
Received: from ERNIE.Berkeley.EDU by SU-AI.ARPA with TCP; 31 May 86 12:33:50 PDT
Received: by ernie.Berkeley.EDU (5.51/1.12)
id AA23586; Sat, 31 May 86 12:32:58 PDT
Date: Sat, 31 May 86 12:32:58 PDT
From: vijay@ernie.berkeley.edu (Vijay Ramamoorthy)
Message-Id: <8605311932.AA23586@ernie.Berkeley.EDU>
To: jmc@SU-AI.ARPA
Subject: AI DISC: Daniel Bobrow
*** Response to Les Dreyfus:
The Dreyfus start their critique by asking for some already occurring
"expectation adjustment". In the excitement engendered by the
enthusiasm of a new adventure, and the relative ease of achieving
superficially interesting results, a number of early researchers made
unwarranted extrapolations of the rate of future progress. As
illustrated by the Dreyfus' documentation of Minsky's statements, major
researchers in the field have a new, and hopefully more realistic
evaluation. It is a lesson for all of us to adjust our immediate
expectations, and not succumb to popular press extrapolation. What is
clear is that there will be no step-function success, with the discovery
of some single philospher's stone; there will be the usual slow
accumulation of knowledge, techniques and technology that will extend
the range of applicability of the science of Artificial Intelligence.
The Dreyfus' also make arguments of impossibility for Artificial
Intelligence based on their characterization of computation. I find
their terms very unclear. The notion of "rule-like" could mean simple
if-then rules (falsely) advertised to enable the building of any expert
system. Or it could refer to the program for any digital computer, from
signal processing, visual feature extraction, and holgraphic
construction to list processing and learning symbolic descriptions from
examples. If they mean to describe the limits of simple if-then rules, I
will join them. I think there we again suffer from press-agentitis with
respect to early success in achieving superficial results. If they
imply limits on computation in general, then a more thorough-going
technical discussion is called for.
I find it strange that use of prototypical cases -- a lot of them -- is
thought to be out of the realm of what could be done with computers.
Minsky made a suggestion for organizing programs that way in his famous
Frames paper over ten years ago. However, it has been easier (less work
for the researchers) to try to extract general rules than to gather
cases, and so we have seen that style flourish. As learning programs
get stronger, and we have mechanisms for bringing large numbers of cases
to computers (as we do with human apprentices), we may see this change.
There is no problem in principle.
Incompatible characterizations of computation are used by the Dreyfuses
for comparing human and machine capabilities. Time for computation is
used for humans: "recognizing two patterns as similar ... seems to be a
direct process for human beings"; but the computer algorithm is
(partially) described: "a logic machine [performs] a complicated process
of first defining each patern in terms of features, ...". Similarly, a
developmental argument ("unlikely in a child's mind") is used to counter
a particular suggested form of algorithm.
If we are to understand the limits of computation, then superficial
comparisons of behavior will not do. Dogmatic statements that
"computers functioning as analytic engines cannot ... recognize the
similarity of whole images" also need explication. In what sense is a
massively parallel machine not an analytic engine, and what similarities
will remain unseen. Can not such machines do local and global
comparisons of the image and find such similarities. Very similar
techniqes are used at the moment in matching points in stereo pairs
(though this is clearly a much simpler task.
After decrying the over-extapolation of results by AI people, the
Dreyfus brothers make a similar error, seeing technical problems as
necessary harbingers of ultimate failure. Hard problems do not "a
degenerating research paradigm" make. However, the Dreyfuses are
probably safe in predicting that the common sense problem will "never"
be solved or that AI will eventually fail, since the work required (we
now believe) is significantly longer than the academic lifetimes of
these predictors.
*** Response to Searle
Searle hides a rabid neo-vitalism under the pretense of making a formal
argument. He gives formal definitions, propositions, and logical
deductions, and challenges one to deny any of the propositions. My
responses are of three kinds -- pointing out the major hole in the
argument, mocking the the form of the polemic, and making a suggestion
as to what is the issue really raised by Searle.
The slippery part of Searle's argument comes between proposition 1 and
proposition 2 in the use of the term syntactic. In 1, he defines
programs (as opposed to computers) as formal and hence syntactic. In
proposition 2, syntax has taken on another meaning. It is put in
contrast (and not equivalent) to semantics. But semantics, as a
"foundational principle behind modern logic ..." is formal. Of course
the syntax of logic is not the semantics of logic, but this is a level
shift rather than a form change. Computers can embody many formal
systems, and transformations between them. Thus computers can embody
such formal semantics. The "inexorability of the syntax/semantics
distinction" is a bad play on words.
But more to the point, what is the meaning of semantics? Is it this
formal study? When we discuss the semantics embodied in minds (as in
proposition 3), we are surely not referring to this meaning of that
word. As Fodor pointed out, minds viewed in this way have no semantics
either. It must be from minds embedded in bodies, and those bodies
embedded in a social system that we get the sense of semantics in which
I can successfully refer to the terminal in front of me (and touch it)--
and have you understand the reference. And isn't this the crux of the
robot argument?
Vitalism again.
Searle seems to believe in Dennett's "wonder tissue" -- that miraculous
aspect of the brain that leads to thought. No separation of function
and structure for him. Although in the beginning of his statement
Searle states:
"I have repeatedly and explicitly denounced [the] view [that] only
carbon-based or perhaps only neuronal-based substances could have the
sorts of thoughts and feelings that humans ... have".
this constrasts with his later statements:
"specific biochemical features of the brain ... cause all of our
mental phenomena" "Does anyone really doubt that all of our mental
states are caused by low level (e.g.neuronal) processes in the brain?"
"Is there someone in AI so totally innocent of biological
knowledge that he thinks that the specfic biochemical powers of human
nervous systems can be duplicated in silicon chips."
This insistence on specific biochemical powers of the brain is
reminiscent of arguments made at the end of the 18th century about how
"organic chemicals" could only be produced in living organisms. It was
only after the test tube synthesis of urea that appropriate properties
of organic chemicals as a natural kind began to be explored. The
backbone of carbon atoms that characterizes what we now call organic
chemicals is found in substances never produced in a living thing, and
in fact poisonous to them.
But the real issue brought in to focus by Searle's arguemnt is how
should we think about the phrase: "have a mind in exactly the same
sense that human beings have minds". To me, the way my wife has a
mind (or how she thinks, as I would put it) is very different from how I
have a mind -- in many dimensions; because she is French by upbringing,
a woman, and a non-techologist to mention a few. Heidigger (among
others) points out how much our thinking (use of our minds) is affected
by the background of our social context and upbringing; he describes the
impossibility of anyone brought up in our culture understanding some
Japanese word. I think that no two creatures will ever have a mind in
the same sense as any other. We must consider many ways of dividing
minds into natural kinds.
The questions we must ask then are how are the similarities and
differences to be characterized. Factors must include task, background,
embodiment,... etc. Dangers include extrapolation from one context to
another. A significant danger is jumping from surface similarities to
projection of other independent capabilities. As Winograd mentions, a
worse danger is that careless use of language to describe current very
limited capabilities of machines can lead to redefinition of the word
intelligence, and perhaps lead us to "systematically undervalue
[certain] aspects of intelligent human action".
Perhaps it is this danger, fear of being judged by the presumed
rationality of computers, that leads Searle to make such strong claims
for the one true way to build minds.
*** Response to Winograd
I looked in vain for the controversial assertions promised by Winograd.
He recommends skepticism about uses of technology, care in our
descriptions of phenomena, and social responsibility of scientists -- I
agree whole-heartedly.
Winograd also worries about "computerization"; will not the use of AI
rather than conventional technology perhaps lead to more flexible (and
hence more "human") systems than most we have to deal with now? This I
believe is the hope of the Fifth Generation computing adventure, despite
its excessive hoopla.
*** Response to Hofstadter
Hofstadter argues for particular aspects of the current AI research
programme, and make predictions of what will be most useful. Only time
will tell what will win in the marketplace of ideas. Hofstadter also
wants us adjust our expectations for the long haul. So be it.
I found Hofstadters rhyme detector most amusing, and the alliteration
detector. Why not postulate going on for a while. Computers don't
count 1,2,3, infinity. Two or three dozen of these might cover the
spectrum (so to speak). Minsky has been proposing (and building
justifications for) an organization of mind based on many such
specialized detectors. They seem not entirely implausible to me either.
*** Response to Rumelhart
Connectionism is an interesting new way of looking at things; it can
probably be usefully guided by attempts to implement symbol processing
(See Touretsky, Symbols among the Neurons, for example), while
preserving good properties of nets. I think both levels of description
will be of significant use in the descripiton of human cognitive
process.